Towards Better Concordance among Contextualized Evaluations in FAST-GDM Problems

Answer Extraction Towards Better Evaluations Of NLP Systems

We argue that reading comprehension tests are not particularly suited for the evaluation of NLP systems. Reading comprehension tests are specifically designed to evaluate human reading skills, and these require vast amounts of world knowledge and common-sense reasoning capabilities. Experience has shown that this kind of full-fledged question answering (QA) over texts from a wide range of domai...

Answer Extraction - Towards better Evaluations of NLP Systems

Towards a better understanding of representing problems as SAT problems

Of fundamental importance for SAT solving is the translation to CNF. One of the basic tasks is to find metrics for determining what are “good” translations, i.e., what makes the resulting SAT problem easy to solve. We introduce a new measure hd(F ), the “hardness”, for formulas F in conjunctive normal form (i.e., clause-sets). hd(F ) for unsatisfiable clause-sets has been studied in [10,12]. Ho...

Making Better Summary Evaluations

This is a position paper which reviews current approaches to the evaluation of automatic summarising systems in particular in the light of older literature targeted at manual abstractors. It identifies the need to improve the handling of novelty in the evaluation of automatically generated summaries and points out the possibility of using existing information retrieval evaluation resources as a...

Towards Better Approaches To Decision Support in Logistics Problems

This position paper outlines our current research to develop a decision support framework for logistics problems. The research covers methodologies, technologies and software tools. The framework considers typical characteristics of real-world logistics problems, common questions to consider in decision support and the technological capabilities to address these characteristics and questions. T...